This repository was archived by the owner on Oct 13, 2025. It is now read-only.
Merge branch 'feature/cluster-2' #253
Merged
+1,845
−895
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR merges the 'feature/cluster-2' branch to enhance data ingestion, add clustering and topic modelling functionality, and refactor ML operations to use PyTorch.
- Updated file reading to support single file, folder, and URL inputs with optional ignore-words filtering.
- Introduced the new ClusterDocs module for document clustering and topic visualizations.
- Refactored MLQRMine to replace Keras-based neural nets with a PyTorch implementation.
Reviewed Changes
Copilot reviewed 26 out of 30 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/qrmine/readfiles.py | Updated file input handling with support for txt/pdf files, folders, and URLs. |
| src/qrmine/nlp_qrmine.py | Minor adjustments updating parameter usage. |
| src/qrmine/mlqrmine.py | Transitioned from Keras to a PyTorch-based neural network implementation. |
| src/qrmine/main.py | Extended CLI options to incorporate new clustering and visualization features. |
| src/qrmine/content.py | Updated content initialization to support language and processing enhancements. |
| src/qrmine/cluster.py | Introduced new clustering module providing LDA topic models and document clustering. |
| pyproject.toml & workflows | Revised dependency definitions and CI/CD configuration updates. |
Files not reviewed (4)
- dev-requirements.in: Language not supported
- dev-requirements.txt: Language not supported
- setup.cfg: Language not supported
- src/qrmine/resources/df_dominant_topic.csv: Language not supported
Comments suppressed due to low confidence (1)
src/qrmine/readfiles.py:42
- Avoid using the built-in name 'input' as a parameter. Consider renaming it to 'source' or 'input_path' for clarity.
def read_file(self, input, comma_separated_ignore_words=None):
return accuracy Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.